Software defect reporting

ABSTRACT

Provided are approaches for software defect reporting. Specifically, one approach provides identifying a software defect; generating a software defect report, wherein the software defect report is generated in real-time as the software defect is identified during testing of the test case, wherein the software defect report is submitted by a testing entity to a software developer responsible for creating a software product having the software defect, and wherein the defect report contains information to identify the location of the software defect in the application code of the software product; determining if the software defect report information is complete; and if the software defect report information is not complete, the updating the defect report information, determining if the software defect is reproducible, and recreating the software defect in the case that the software defect is reproducible.

CROSS-REFERENCE TO RELATED APPLICATION

This patent document is a continuation of, and claims the benefit of,co-pending and co-owned U.S. patent application Ser. No. 14/330,187,filed Jul. 14, 2014, which is a continuation of commonly owned U.S.patent application Ser. No. 12/759,829, filed Apr. 14, 2010, issued Aug.19, 2014 as U.S. Pat. No. 8,813,039, the entire contents of which areherein incorporated by reference.

FIELD OF THE INVENTION

Aspects of the present invention provides for a method and a system forimproving the defect reporting mechanism in the software qualityautomated process using automated log scanning.

BACKGROUND OF THE INVENTION

A software bug, or defect, is the common term used to describe an error,flaw, mistake, failure or fault in a computer program or system thatproduces an incorrect or unexpected result or causes it to behave inunintended ways. Most bugs arise from mistakes and errors made by peoplein either a program's source code or its design. Others may be caused bycompilers producing incorrect code.

Bugs can have a wide variety of effects with varying levels ofinconvenience to the user of the program. Some bugs have only a subtleeffect on the program's functionality and may thus lie undetected forlong time while more serious bugs may cause the program to crash orfreeze. Other bugs may have more serious ramifications such as securitybugs that might for example enable a malicious user to By-pass accesscontrols in order to obtain unauthorized privileges.

The risk (likelihood of occurrence and impact) of software bugs isimmense. Virtually every business in the United States and across theglobe depends on software for the development, production, distributionand support of products and services. Entire industries alone have beenenabled by low-cost computational capabilities supplied by computers andsoftware.

According to market researcher DataMonitor, the size of the worldwidesoftware industry in 2008 was US$ 303.8 billion, an increase of 6.5%compared to 2007. Americas account for 42.6% of the global softwaremarket's value. DataMonitor forecasts that in 2013, the global softwaremarket will have a value of US$ 457 billion, an increase of 50.5% since2008.

In 2002, a study commissioned by the US Department of Commerce'sNational Institute of Standards and Technology concluded that softwarebugs, or errors, are so prevalent and so detrimental that they cost theUS economy an estimated $59 billion annually, or about 0.6 percent ofthe gross domestic product. The Department also concluded that at thenational level over half the costs of software bugs are borne bysoftware users and the remainder by the software developers/vendors. Thestudy also found that, although all errors cannot be removed, more thana third of these costs, or an estimated $22.2 billion, could beeliminated by an improved testing infrastructure that enables earlierand more effective identification and removal of software defects.

A number of inventions have been established that attempt to improve thequality of software. Many of these prescribe techniques for identifyingdefects earlier in the development cycle, automating the bug, or defect,identification process or planning through predicting modeling. Howeverthese approaches focus purely on the planning, management and preventionof software defects. They fail, however, to address the potentially moreimportant issue of how to adequately address defects once they have beenidentified.

SUMMARY OF THE INVENTION

The present invention ties together the identification of a defectduring testing with the supporting application code on the systems thatled to the generation of the defect. The association between the two isthrough the use of a unique trace identification, or Trace ID. ThisTrace ID is unique to each interaction between the consumer (e.g.,client browser) and the provider system(s) (web application servers,back office systems, etc.). When a potential defect is identified by theend user or tester, the present invention will “collect” the associatedlog messages using the unique Trace ID and attaches those to the defect.The software developer providing the defect triage can then readilyreview the associated log files without having to manually search thesupporting log files manually.

One embodiment of the present invention is a method comprising: acomputer identifying a software defect; the computer generating asoftware defect report, wherein the software defect report is generatedin real-time as the software defect is identified during testing of thetest case, wherein the software defect report is submitted by a testingentity to a software developer responsible for creating a softwareproduct having the software defect, and wherein the defect reportcontains information to identify the location of the software defect inthe application code of the software product; the computer determiningif the software defect report information is complete; and if thesoftware defect report information is not complete, the computerupdating the defect report information, determining if the softwaredefect is reproducible, and recreating the software defect in the casethat the software defect is reproducible.

Another embodiment of the present invention is a computer systemcomprising: a memory medium comprising program instructions; a buscoupled to the memory medium; and a processor, for executing the programinstructions, coupled to an input controller via the bus that whenexecuting the program instructions causes the system to: identify asoftware defect; generate a software defect report, wherein the softwaredefect report is generated in real-time as the software defect isidentified during testing of the test case, wherein the software defectreport is submitted by a testing entity to a software developerresponsible for creating a software product having the software defect,and wherein the defect report contains information to identify thelocation of the software defect in the application code of the softwareproduct; determine if the software defect report information iscomplete; and if the software defect report information is not complete,the computer updating the defect report information, determining if thesoftware defect is reproducible, and recreating the software defect inthe case that the software defect is reproducible.

Yet another embodiment of the present invention is a computer programproduct comprising a computer readable storage medium, and programinstructions stored on the computer readable storage medium, to:identify a software defect; generate a software defect report, whereinthe software defect report is generated in real-time as the softwaredefect is identified during testing of the test case, wherein thesoftware defect report is submitted by a testing entity to a softwaredeveloper responsible for creating a software product having thesoftware defect, and wherein the defect report contains information toidentify the location of the software defect in the application code ofthe software product; determine if the software defect reportinformation is complete; and if the software defect report informationis not complete, the computer updating the defect report information,determining if the software defect is reproducible, and recreating thesoftware defect in the case that the software defect is reproducible.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of this invention will be more readilyunderstood from the following detailed description of the variousaspects of the invention taken in conjunction with the accompanyingdrawings in which:

FIG. 1 shows a data processing system suitable for implementing anembodiment of the present invention.

FIG. 2 shows a network which would work with an embodiment of thepresent invention.

FIG. 3 illustrates an embodiment of a method of an “As-Is IdentificationProcess” wherein the defect is identified during the testing phase.

FIG. 4 illustrates an embodiment of a method of an “As-Is Fix Process”.

FIG. 5 illustrates an embodiment of a method of a “To-Be Defect FixProcess” of the present invention.

FIG. 6 illustrates an embodiment of a system showing interaction betweenthe client and the server of the present invention.

FIG. 7 illustrates an embodiment of a system showing a log consolidatorinterceptor and various systems.

FIG. 8 illustrates a sample application screen that may result in anerror in a transactional application.

The drawings are merely schematic representations, not intended toportray specific parameters of the invention. The drawings are intendedto depict only typical embodiments of the invention, and thereforeshould not be considered as limiting the scope of the invention.

DETAILED DESCRIPTION OF THE DRAWINGS

The present invention, which meets the needs identified above, is amethod and system for decreasing the overall time required to resolve asoftware defect by providing the software developer with more completeand timelier information obtained at the point the defect is identified.The present invention enhances the typical data attached to the defectso that the analysis and discovery period of the defect is reduced andthe overall time for defect resolution is reduced. The enhanced dataincludes details about the transaction being executed and any exceptionstack traces generated during the execution of the transaction.

The present invention has several advantages over previously mentionedprior systems. While other systems attempt to improve the defectidentification process, this invention attempts to decrease the overallresponse time to deliver a fix. By automating activities that wouldnormally be performed manually, the time from initial defectidentification to defect resolution may be significantly reduced byseveral factors.

Typically, once a defect is identified, logged, verified and deliveredto the software developer, the developer will then use the baseinformation submitted by the tester or end user as a starting point forhis/her analysis. The goal of the analysis step is to identify wherewithin the code the defect originated. The analysis may follow one ofseveral process paths depending on what point in the product lifecyclethe defect was identified (e.g., development, testing, production,etc.), what level of information was included (e.g., steps to reproducethe defect), and whether the user has the ability to recreate the defect(i.e., the defect may arise only in certain circumstances that may notbe easily reproducible). Regardless, in a best case scenario, thisanalysis step indeed a costly part of the defect resolution process.

The present invention ties together the identification of a defectduring testing with the supporting application code on the systems thatled to the generation of the defect. The association between the two isthrough the use of a unique trace identification, or Trace ID. ThisTrace ID is unique to each interaction between the consumer (e.g.,client browser) and the provider system(s) (web application server, backoffice systems, etc). When a potential defect is identified by the enduser or tester, the present invention will “collect” the associated logmessages using the unique Trace ID and attaches those to the defect. Thesoftware developer providing the defect triage can then readily reviewthe associated log files without having to manually search thesupporting log files manually.

One advantage of this invention is that it is concerned with providingthe relevant information at the point of appearance of the defect. Bygathering this information real-time, the need to reproduce the defectin a development or test environment after the fact is significantlyreduced. This eliminates a significant costly step that is part of thetypical development defect fix lifecycle today. As systems become morecomplex and more widespread, the present invention will play asignificant role in improving the overall quality of software.

A data processing system 100, such as system 102 shown in FIG. 1,suitable for storing and/or executing program code of the presentinvention will include a computer system 104 having at least oneprocessor (processing unit 106) coupled directly or indirectly to memoryelements through a system bus 112. Computer system 104 may be a client,e.g., a browser, or a server. The memory elements can include localmemory (RAM 130) employed during actual execution of the program code,bulk storage (storage 118), and cache memories (cache 132) that providetemporary storage of at least some program code in order to reduce thenumber of times code must be retrieved from bulk storage 118 duringexecution. Input/output or I/O devices (external devices 116) (includingbut not limited to keyboards, displays (display 120), pointing devices,etc.) can be coupled to the system either directly or throughintervening I/O controllers (I/O interface(s) 114).

Network adapters (network adapter 138) may also be coupled to the system200 to enable the data processing system (as shown in FIG. 2, dataprocessing unit 202) to become coupled through network connections(network connection 208) to other data processing systems (dataprocessing unit 204), remote printers (printer 212) and/or storagedevices (storage 214) through intervening private and/or public networks(network 210).

FIG. 3 illustrates an “As-Is Defect Identification Process” 300 foridentifying defects during a testing phase of a software product havinga test lead 302, a tester 304 and a software developer 306. FIG. 3highlights a typical workflow executed during the testing phase of asoftware component. In this scenario, the software developer 306releases software to tester 304 at 308, then tester 304 executes a testcase at 310 in order to satisfy a particular test strategy objective. Atest case is a pre-defined series of steps designed to satisfy aparticular testing objective (e.g., functional verification of businessrequirement). A properly documented test case may contain basicinformation about the purpose of the test case, execution steps,preconditions and expected results (see sample test case). If the actualresults or behavior observed during execution varies from the expectedresults, it is determined whether a defect is found at 311. If a defectis found at 311, a defect report is prepared at 313 and the defectreport will be logged in a defect repository or other tracking mechanismat 314, the defect is passed to the test lead 302 at 318, where it isdetermined whether the defect was “valid” and, if not, the defect testis updated at 320 or, if so, the defect is fixed at 322. After the testcase is updated at 320, the defect is fixed at 322. The logging of thedefect report is of immediate concern to the present invention.Reporting mechanisms require limited information about the identifieddefect. For example, in a simple model the defect may simply contain areference to the test case and the actual results that differed from theexpected results.

FIG. 4 highlights a workflow, or method, 400 for an “As-Is Defect FixProcess” executed after the defect has been identified and logged. Thisprocess highlights a process 400 that a software developer 402 wouldexecute when a bug, or defect, is received in his/her queue. There aresource system developers 404 and a tester 406 in this process as well.At 408, the software developer 402 fixes a software defect, reviews adefect report at 410, determines whether the defect report informationis complete at 412. If the defect information is complete, the defectreport is updated at 416 and the process recycles at 410. If the defectinformation is incomplete, at 414, software developer 402 determineswhether the defect is reproducible and, if not, tester 406 coordinates adefect retest at 418 and recreates the defect at 420. If softwaredeveloper 402 determines that the defect is reproducible, softwaredeveloper 402 recreates the defect at 420, searches available log filesat 422, and determines if there is log data available at 428 and, if so,it is released at 436. If there is no log data available, log data filesare analyzed at 430. If there are other systems involved at 424, sourcesystem developers 404 search available log files at 426 and determinewhether there is log data available at 428 and the same process asdescribed above as described above is followed. At 432, softwaredeveloper 402 determines the defect source. At 434, the defect is fixedand, at 436, the software is released. In this process, the majority ofsteps (408-430 as shown in the figure) are focused on gathering thenecessary information to understand the source of the defect (432 asshown in the figure). That is, identifying and locating the point in thesource code that is either coded incorrectly or contains the flawedlogic.

Steps 408-430, as shown in the figure, are a significant contributor tothe overall defect timeline. Gathering the necessary facts to understandwhat the user was doing that generated the defects can typically take upto 50% of the end-to-end process. There are several points in thisprocess that can severely impede the software developer's ability todiagnose a defect root cause. For example, the identified defect may notcontain the complete set of information for the developer to adequatelyunderstand the problem at hand (412). The defect may not bereproducible. Many transactional systems, for example, have complex datapre-conditional requirements. In addition, the systems themselves maynot be “available” for the software developer to leverage. Test systemstypically have restricted access in order to avoid corruption of testdata. The third major hindrance may be when multiple systems areinvolved (424). In this scenario, the defect may arise by any of thesystems involved with satisfying the specific user transaction. Thismulti-system scenario significantly increases the complexity in purelygathering the necessary log information to diagnose the problem.

FIG. 5 highlights a proposed “To-Be Defect Process” workflow 500 thatwould be realized as part of the present invention having a softwaredeveloper 502 and a tester 504. In this scenario, the steps between thetime the defect is received by the software developer (506) to the timethat the defect root cause is understood (514) is dramaticallysimplified. In addition, the overall timeframe would be reduced. Thepresent invention expedites this process because the necessaryinformation to perform the initial root cause analysis is captured atthe point of defect identification and attached to the defect reportsubmitted by the tester. This is different than the current process thatrequires a significant amount of manual user intervention.

FIG. 5 illustrates a process that begins at 506 where a defect isdetected by software developer 502 and moves to 508 where softwaredeveloper 502 reviews the defect report. At 510, software developer 502determines whether the information in the defect report is complete and,if not, tester 504 updates the information in the defect report. But, ifsoftware developer 502 determines that the information in the defectreport is complete, software developer 502 analyzes log entries, orfiles, at 512, determines the defect source at 514, fixes the defect at516 and releases the software at 518.

FIG. 6 highlights the use of a standard Interceptor pattern 600 toenable the present invention in a typical software application. It showsa service/dispatcher 602 and an action proxy 604. In this, twoInterceptors are leveraged: the Trace ID Interceptor 606 and a LogConsolidator Interceptor 608. Trace ID Interceptor 606 is responsiblefor intercepting outgoing results from service/dispatcher 602 andassigning a unique identifier. This unique identifier, or Trace ID, willthen function as the primary key for the next request from the client tothe backend systems. When a failure occurs, the Trace ID will be used toretrieve the pertinent information from the providing systems.

Log Consolidator Interceptor 608 is responsible for interceptingoutgoing results and inspecting for the presence of errors. Should anerror exist, Log Consolidator Interceptor 608 will create a result 612and make a series of calls (Action 610) to the relevant source system toobtain the log details. This may be obtained asynchronously so as not tohave a performance impact on the return call to the consumingapplication.

As shown in FIG. 7, a system 700 has a Log Consolidator Interceptor 702connected to systems 1 (704), 2 (706) and 3 (708). Each of the systems 1(704), 2 (706) and 3 (708) have their own respective GetLogs 710 thatare connected to log data 712. When the presence of errors is detected,the Log Consolidator Interceptor 702 will make calls to the sourcesystems requesting the associated log information be obtained. Thesource systems will then search the available logs to obtain the traceinformation.

FIG. 8 illustrates a sample web application screen 800 that may resultin a error in a transactional application. In this figure, the userspecifies a series of information (update user profile 802), such asuser name (804, 806), address (808, 810, 812, 814) and then clicks an‘OK’ button (816) or otherwise presses a ‘Cancel’ button (818). If the‘OK’ button is pressed, the data will then be posted to the server-sidecomponent for processing.

It should be understood that the present invention is typicallycomputer-implemented via hardware and/or software. As such, clientsystems and/or servers will include computerized components as known inthe art. Such components typically include (among others) a processingunit, a memory, a bus, input/output (I/O) interfaces, external devices,etc.

While shown and described herein as a system and method for improvingthe defect reporting mechanism in the software quality automated processusing automated log scanning, it is understood that the inventionfurther provides various alternative embodiments. For example, in oneembodiment, the invention provides a computer-readable/useable mediumthat includes computer program code to enable a system for improving thedefect reporting mechanism in the software quality automated processusing automated log scanning. To this extent, thecomputer-readable/useable medium includes program code that implementseach of the various process steps of the invention. It is understoodthat the terms computer-readable medium or computer useable mediumcomprises one or more of any type of physical embodiment of the programcode. In particular, the computer-readable/useable medium can compriseprogram code embodied on one or more portable storage articles ofmanufacture (e.g., a compact disc, a magnetic disk, a tape, etc.), onone or more data storage portions of a computing device, such as memoryand/or storage system (e.g., a fixed disk, a read-only memory, a randomaccess memory, a cache memory, etc.), and/or as a data signal (e.g., apropagated signal) traveling over a network (e.g., during awired/wireless electronic distribution of the program code).

In another embodiment, the invention provides a computer-implementedmethod for improving the defect reporting mechanism in the softwarequality automated process using automated log scanning. In this case, acomputerized infrastructure can be provided and one or more systems forperforming the process steps of the invention can be obtained (e.g.,created, purchased, used, modified, etc.) and deployed to thecomputerized infrastructure. To this extent, the deployment of a systemcan comprise one or more of (1) installing program code on a computingdevice, such as computer system from a computer-readable medium; (2)adding one or more computing devices to the computer infrastructure; and(3) incorporating and/or modifying one or more existing systems of thecomputer infrastructure to enable the computerized infrastructure toperform the process steps of the invention.

As used herein, it is understood that the terms “program code” and“computer program code” are synonymous and may mean any expression, inany language, code or notation, of a set of instructions intended tocause a computing device having an information processing capability toperform a particular function either directly before or after either orboth of the following: (a) conversion to another language, code ornotation; and/or (b) reproduction in a different material form. To thisextent, program code can be embodied as one or more of: anapplication/software program, component software/a library of functions,an operating system, a basic I/O system/driver for a particularcomputing and/or I/O device, and the like.

In another embodiment, the invention provides a business method thatperforms the process steps of the invention on a subscription,advertising, and/or fee basis. That is, a service provider, such as aSolution Integrator, could offer to deploy a computer infrastructure forimproving the defect reporting mechanism in the software qualityautomated process using automated log scanning. In this case, theservice provider can create, maintain, and support, etc., the computerinfrastructure by integrating computer-readable code into a computingsystem, wherein the code in combination with the computing system iscapable of performing the process steps of the invention for one or morecustomers. In return, the service provider can receive payment from thecustomer(s) under a subscription and/or fee agreement and/or the serviceprovider can receive payment from the sale of advertising content to oneor more third parties.

The foregoing description of various aspects of the invention has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formdisclosed, and obviously, many modifications and variations arepossible. Such modifications and variations that may be apparent to aperson skilled in the art are intended to be included within the scopeof the invention as defined by the accompanying claims.

What is claimed is:
 1. A method comprising: assigning to eachinteraction with a software product being tested a unique trace ID; acomputer, in response to identifying a software defect, automaticallyassociating the unique trace ID corresponding to the interaction duringwhich the software defect occurred with log messages corresponding tothe interaction; the computer, in response to identifying the softwaredefect, generating a software defect report based on the unique traceID, wherein the defect report contains the log messages collected usingthe unique trace ID that associates the software defect during executionof a testing case with application code on a source system that led to ageneration of the defect to identify a location of the software defectin the application code of the software product.
 2. The method asdefined in claim 1 further comprising the computer analyzing log filesin the case that the software defect report information is complete. 3.The method as defined in claim 2 further comprising, if the defectreport information is complete, searching log files, determining if logfiles are available, and releasing the software.
 4. The method asdefined in claim 2 further comprising: the computer determining a defectsource; the computer fixing the software defect; and the computerreleasing the software.
 5. A computer system comprising: a memory mediumcomprising program instructions; a bus coupled to the memory medium; anda processor, for executing the program instructions, coupled to an inputcontroller via the bus that when executing the program instructionscauses the system to: assign to each interaction with a software productbeing tested a unique trace ID; automatically associate, in response toidentifying a software defect, the unique trace ID corresponding to theinteraction during which the software defect occurred with log messagescorresponding to the interaction; the computer, in response toidentifying the software defect, generating a software defect reportbased on the unique trace ID, wherein the defect report contains the logmessages collected using the unique trace ID that associates thesoftware defect during execution of a testing case with application codeon a source system that led to a generation of the defect to identify alocation of the software defect in the application code of the softwareproduct.
 6. The computer system as defined in claim 5, the programinstructions further causing the system to analyze log files in the casethat the software defect report information is complete.
 7. The computersystem as defined in claim 6, wherein if the defect report informationis complete, the program instructions further causing the system tosearch log files, determine if log files are available, and release thesoftware.
 8. The computer system as defined in claim 6, the programinstructions further causing the system to: determine a defect source;fix the software defect; and release the software.
 9. A computer programproduct comprising a computer readable storage medium, and programinstructions stored on the computer readable storage medium, to: assignto each interaction with a software product being tested a unique traceID; automatically associate, in response to identifying a softwaredefect, the unique trace ID corresponding to the interaction duringwhich the software defect occurred with log messages corresponding tothe interaction; the computer, in response to identifying the softwaredefect, generating a software defect report based on the unique traceID, wherein the defect report contains the log messages collected usingthe unique trace ID that associates the software defect during executionof a testing case with application code on a source system that led to ageneration of the defect to identify a location of the software defectin the application code of the software product.
 10. The computerprogram product as defined in claim 9, the program instructions furthercausing the system to analyze log files in the case that the softwaredefect report information is complete.
 11. The computer program productas defined in claim 9, wherein if the defect report information iscomplete, the program instructions further causing the system to searchlog files, determine if log files are available, and release thesoftware.
 12. The computer program product as defined in claim 9, theprogram instructions further causing the system to: determine a defectsource; fix the software defect; and release the software.